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DETAILED ACTION 

1 . This office action is responsive to communication filed on June 1 1 , 2009. Claims 
16 and 17 are pending in the application and have been examined by the Examiner. 

Information Disclosure Statement 

2. The Information Disclosure Statement filed July 6, 2009 was received and has 
been considered by the Examiner. 

Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 
1 1 , 2009 has been entered. 

Response to Arguments 

4. Applicant's arguments filed June 1 1 , 2009 have been fully considered but they 
are not persuasive. 

5. Applicant argues that Nakamura is not simply missing the first and second areas 
of memory out of the elements of claim 1 6 as amended, i.e., it lacks more elements of 
claim 16. For example, since Nakamura lacks the first and second memory areas, it 
automatically fails to teach the control schemes as recited in amended claim 16 
discussed above, e.g., the control schemes indicated as (a) and (b), i.e., (a) said image 
processing device processes a color space conversion for the first RAW data readout 
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from said first area in accordance with start of reading tlie second RAW data from tlie 
image sensing element in the second image sensing operation, and (b) the white 
balance integral processing for the second RAW data by said white balance integration 
device and the color space conversion for first RAW data by said image processing 
device processes are performed in parallel during reading of the second RAW data from 
the image sensing element. 

6. The Examiner respectfully disagrees. In response to applicant's arguments 
against the references individually, one cannot show nonobviousness by attacking 
references individually where the rejections are based on combinations of references. 
See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 
F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

7. As per (a), Nakamura teaches that the image processing device processes a 
color space conversion for the first RAW data readout from said first area during "Image 
processing 1", "Pc" of figure 8, column 7, lines 24-33. Nakamura teaches the start of 
reading the second RAW data from the image sensing element in the second image 
sensing operation during period "Pe" of figure 8, column 7, lines 41-44. The Examiner 
agrees that these processes of Nakamura are not performed in parallel. However, 
Anderson teaches that one buffer in memory is filled with image data (i.e. the reading of 
second RAW data from an image sensing element) in parallel with the processing of 
previous captured image data output from a second buffer (i.e. image processing for the 
first RAW data readout from said first area), column 6, lines 47-50. Anderson teaches 
that said image processing includes color space conversion (column 6, lines 19-23). 
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Therefore, the combination of Nakamura and Anderson teaches said image processing 
device processes a color space conversion for the first RAW data readout from said first 
area in accordance with start of reading the second RAW data from the image sensing 
element in the second image sensing operation. 

8. As per (b), Nakamura teaches the white balance integral processing for the 
second RAW data by said white balance integration device is performed in accordance 
with start of reading the second RAW data during period "Pe" of figure 8, column 7, lines 
41-44. Nakamura teaches the color space conversion for first RAW data by said image 
processing device during "Image processing 1", "Pc" of figure 8, column 7, lines 24-33. 
The Examiner agrees that Nakamura does not teach that these processes are 
performed in parallel during reading of the second RAW data from the image sensing 
element. However, Anderson teaches that one buffer in memory is filled with image 
data (i.e. the reading of second RAW data from an image sensing element) in parallel 
with the processing of previous captured image data output from a second buffer (i.e. 
image processing for the first RAW data readout from said first area), column 6, lines 
47-50. Anderson teaches that said image processing includes color space conversion 
(column 6, lines 19-23). Therefore, the combination of Nakamura and Anderson 
teaches the white balance integral processing for the second RAW data by said white 
balance integration device and the color space conversion for first RAW data by said 
image processing device processes are performed in parallel during reading of the 
second RAW data from the image sensing element. 
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9. Applicant argues that referring to Fig. 8 of Nakamura as shown below, it appears 
that the Examiner tries to equate the "IMAGE PROCESSING 1" (Pc) to the control 
scheme indicated as (a) above. However, the "EXPOSURE/STORAGE 2" is not an 
equivalent process to the "reading the second RAW data read from the image sensing 
element and stored In the second area of the memory," as recited in amended claim 16. 
For example, the output of the "EXPOSURE/STORAGE 2" process is not stored in a 
second area of the memory as required by amended claim 16. 

1 0. The Examiner respectfully disagrees. In the office action mailed March 1 1 , 2009, 
the Examiner stated (see page 7), "Nakamura et al. further does not explicitly teach that 
the processing device processes a color space conversion for the first RAW data 
readout from said first area in accordance with start of reading the second RAW data 
from the image sensing element." On the contrary, Nakamura teaches that the "Image 
Processing 1" ("Pc", figure 8) and the reading out of the second RAW data (see "Pe") 
are not performed in parallel. However, Anderson teaches that one buffer in memory is 
filled with image data (i.e. the reading of second RAW data from an image sensing 
element) in parallel with the processing of previous captured image data output from a 
second buffer (i.e. image processing for the first RAW data readout from said first area), 
column 6, lines 47-50. Anderson teaches that said image processing includes color 
space conversion (column 6, lines 19-23). Therefore, the combination of Nakamura and 
Anderson teaches that the storage of the second RAW data and the processing for first 
RAW data by said image processing device processes are performed in parallel during 
reading of the second RAW data from the image sensing element. 
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1 1 . Applicant argues that Nakamura does not disclose or suggest that it is limited to 
display the live view image in a condition that the integral processing for the second 
RAW data is finished but the color space conversion processing for the first RAW data 
is not finished and that it is allowed to display the live view image in response to an 
operation that the color space conversion processing for the first RAW data is finished 
after the integral processing for the second RAW data is finished, as specifically recited 
in amended claim 16. Additionally, Applicant argues that Nakamura fails to teach that 
said while balance calculation device calculates white balance coefficient of the second 
RAW data after said display device starts to display the live view image. 

12. The Examiner respectfully disagrees. In response to applicant's arguments 
against the references individually, one cannot show nonobviousness by attacking 
references individually where the rejections are based on combinations of references. 
See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 
F.2d 1091 , 231 USPQ 375 (Fed. Cir. 1986). Anderson teaches that a live view image is 
always displayed. For instance, during the alternate image storage and processing of 
input buffers A and B of figure 4B, frames are alternately output from frame buffers A 
and B, column 6, lines 47-56. Because images are output from a frame buffer to the 
display, the display time does not rely on any image processing period of time. 
Therefore, the combination of Nakamura, Anderson and Taniguchi et al. teaches that 
the display is limited to display the live view image (i.e. the display must display a live 
view image) in a condition that the integral processing for the second RAW data is 
finished but the color space conversion processing for the first RAW data is not finished 
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(the display always displays a live view image, see Anderson above) and that it is 
allowed to display the live view image (i.e. the live view image is displayed) in response 
to an operation that the color space conversion processing for the first RAW data is 
finished after the integral processing for the second RAW data is finished (the display 
always displays a live view image, see Anderson above), as specifically recited in 
amended claim 16. An object image is continuously displayed on the display as taught 
by Anderson, including any time when a white balance coefficient is calculated. 

1 3. Applicant argues that there is no motivation to modify the reference of combine 
reference teachings as proposed by the Examiner. 

14. In response to applicant's argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed invention 
where there is some teaching, suggestion, or motivation to do so found either in the 
references themselves or in the knowledge generally available to one of ordinary skill in 
the art. See In re Fine, 837 F.2d 1071 , 5 USPQ2d 1596 (Fed. Cir. 1988)and In re 
Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, Anderson 
teaches the explicit motivation of improving the display speed of the digital camera and 
preventing the tearing of the image in the display (column 6, line 65 through column 7, 
line 3). Taniguchi et al. teaches the explicitly motivation of performing white balance 
without any erroneous adjustment and adverse influence of a high chromaticity region of 
the image (column 2, lines 40-50). 

15. Therefore, the rejection is maintained by the Examiner. 
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Claim Objections 

16. Claim 16 is objected to because of tine following informalities: Lack of clarity and 
precision. 

17. Claim 16 recites,"said display device is limited to display the live view image 
during the integral processing for the second RAW data is finished but the color 
space conversion processing for the first RAW data is not finished, said display device 
is allowed to display the live view image in response that the color space conversion 
processing for the first RAW data is finished after the integral processing for the 
second RAW data is finished". The above-cited passage is not written in proper 
idiomatic English and thus lacks clarity. Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

1 8. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

19. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 

al. (US 5,619,347). 

20. The Examiner's response to Applicant's arguments, as outlined above, in hereby 
incorporated into the rejection of claim 16 by reference. 

Consider claim 16, Nakamura et al. teaches: 
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An image sensing apparatus ("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device ("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element as RAW data (column 3, lines 50-58); 

a memory ("DRAM", 232, figure 4) which has a first area for temporarily storing 
first RAW data obtained in a first image sensing operation of said image sensing device 
and for temporarily storing second RAW data obtained in a second image sensing 
operation next to the first image sensing operation of said image sensing device (See 
figure 7, column 7, lines 10-49. First raw image data is written into the DRAM (232) 
over channel 1 . During a subsequent image capture operation, this first raw data is 
read out of DRAM (232) over channel 2 for image processing, as second image data is 
written into the DRAM (232) over channel 1 .); 

a white balance integration device (21 1a, figure 6A) which integrates at least one 
of the first and second RAW data for a white balance processing (The white balance 
integration device (21 1 a) is part of the image signal processor (21 1 , figure 4). The 
RAW image data is subjected to white balance processing, and then stored into the 
DRAM (232), column 6, lines 21-26.); 

an image processing device (21 1 , figure 4) which performs image processing of 
the first and second RAW data readout from said memory (The image processing 
device (21 1) performs processing such as color space conversion on the RAW data 
readout from memory (232), column 4, lines 8-10, column 6, lines 26-36, column 7, lines 
24-33.), 
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a display device ("EVF", 20, or "LCD", 10, figure 4) wliicli displays a live view 
image during imaging on the image sensing element (The display acts as a "live view 
display" (i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control device ("main CPU", 21, figure 4) which controls said memory (232), 
said white balance integration device (211a), said image processing device (211), and 
said display device (20, 10, column 4, lines 1-27. The main CPU (21) comprises the 
image processing device (21 1) which contains the white balance integration device 
(21 la), a bus controller (218) for controlling the memory (232), and a video encoder 
(213) which supplies analog image signals to the display.), 

wherein, said control device (21) controls so that, said image processing device 
(21 1 ) processes a color space conversion for the first RAW data readout from said 
memory (232) in accordance with start of exposure/storage of the second RAW data 
from the image sensing element (303) in the second image sensing operation (See 
figure 8, column 7, lines 24-33. A frame of raw data obtained by an immediately 
preceding image (i.e. first image data) is read out of DRAM (232) and subjected to color 
space conversion while second image data is obtained through exposure and storage of 
the CCD.), and said display device ("LCD", 10) displays the object image (See "Live 
View Display" on the right side of figure 8.) after the color space conversion processing 
for the first RAW data (See "Pc", figure 8) and the integral processing (See Readout 2, 
"Pe", figure 8) for the second RAW data (See column 7, lines 24-49. The LCD exhibits 
a "Live View Display" after the color space conversion "Pc" and integral processing 
"Pe".). 
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However, Nakamura et al. does not explicitly teach that said memory (232) has a 
second area for storing the second RAW image data. Nakamura et al. further does not 
explicitly teach that the processing device processes a color space conversion for the 
first RAW data readout from said first area in accordance with start of reading the 
second RAW data from the image sensing element. Nakamura et al. does not explicitly 
teach that the display device is limited to display the live view image in a condition that 
the integral processing for the second RAW data is finished but the color space 
conversion processing for the first RAW data is not finished and that it is allowed to 
display the live view image in response to an operation that the color space conversion 
processing for the first RAW data is finished after the integral processing for the second 
RAW data is finished 

Anderson is similar to Nakamura et al. in that Anderson teaches of a camera 
(figures 1-3) with a memory (figure 4a). Anderson also similarly teaches of reading out 
raw image data from an image sensor (1 14, figure 1 , column 5, lines 59-64), storing the 
data in a memory (530, column 5, line 59 through column 6, line 3), and subsequently 
performing color space conversion on the image data (column 8, line 59 through column 
9, line 7). 

However, in addition to Nakamura et al., Anderson teaches that said memory 
(figures 4a and 4b) has a second area (Input buffer, 2, B) for storing the second RAW 
image data, each of the first and second areas of memory stores the RAW data from 
the first and second operations alternately, that said memory stores the third RAW data 
in the first area in which the color space conversion of the first RAW data by said 
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image processing device is finislied, and tliat the processing device processes a color 
space conversion for the first RAW data readout from said first area in accordance with 
start of reading the second RAW data from the image sensing element (See figure 4a, 
column 4, line 59 through column 6, line 3, column 6, lines 38-56, column 8, line 59 
through column 9, line 8. Anderson teaches, "Referring again to FIG. 4B, the ping- 
pong buffers are utilized during live view mode as follows. While input buffer A is filled 
with image data, the data from input buffer B is processed and transmitted to frame 
buffer B. At the same time, previously processed data in frame buffer A is output to the 
LCD screen 402 for display. While input buffer B is filled with image data, the data 
from input buffer A is processed and transmitted to frame buffer A. At the same time, 
previously processed data in frame buffer B is output to the LCD screen 402 for 
display." As one buffer (i.e. the second area) is filled with raw image data (i.e. the start 
of reading of the second RAW data from the image sensing element), the other buffer 
(i.e. the first area) is emptied and processed (i.e. first RAW data is readout from the 
first area), which processing involves color space conversion (See 612, figure 7, 
column 8, line 59 through column 9, line 8). Anderson teaches that the input buffers A 
and B alternate between an input cycle and a processing cycle, column 6, lines 8-10. 
Therefore, buffer A is an input buffer during a first phase, an output buffer during a 
second phase when second RAW data is written into buffer B, and again an input 
buffer during a third phase when third RAW data is output from the sensor an into 
memory. Thus, the third image data is stored in the same area (i.e. buffer A) as the 
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first image data during the third phase, after the first image data is processed during 
the second phases- 
Anderson teaches that a live view image is always displayed. For instance, 
during the alternate image storage and processing of input buffers A and B of figure 
4B, frames are alternately output from frame buffers A and B, column 6, lines 47-56. 
Because images are output from a frame buffer to the display, the display time does 
not rely on any image processing period of time. Therefore, the combination of 
Nakamura and Anderson teaches that the display is limited to display the live view 
image (i.e. the display must display a live view image) in a condition that the integral 
processing for the second RAW data is finished but the color space conversion 
processing for the first RAW data is not finished (the display always displays a live view 
image, see Anderson above) and that it is allowed to display the live view image (i.e. 
the live view image is displayed) in response to an operation that the color space 
conversion processing for the first RAW data is finished after the integral processing for 
the second RAW data is finished (the display always displays a live view image, see 
Anderson above). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use ping-pong buffers as taught by Anderson in the 
camera taught by Nakamura et al. to read out raw image data from the image sensor 
concurrent with the processing color space conversion of previous image data, for the 
benefit of improving the display speed of the digital camera and preventing the tearing 
of the image on the display (Anderson, column 5, line 65 through column 6, line 3.). 
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Nakamura et al. alone does not explicitly teach that the integral processing 
for the second RAW data by said white balance integration device and the color 
space conversion for first RAW data by said image processing device processes 
are performed in parallel during reading of the second RAW data from the image 
sensing element. However, because the white balance integration taught by 
Nakamura et al. is performed during the raw data writing ("Pe"), and thus in 
parallel with the readout of the second image data (see figure 8, column 7, lines 
41^9), the combination of Nakamura et al. and Anderson teaches that that 
integral processing of the second image data and the color space conversion of 
the first image data are performed in parallel. This is because Anderson modifies 
Nakamura et al. such that the image processing of the first image data (which 
includes color space conversion) readout from the first area of memory takes 
place concurrentiy with the readout of second RAW image data from the image 
sensor (which includes integration) and into the second area of memory, as 
discussed above. 

However, the combination of Nakamura et al. and Anderson does not explicitly 
teach a white balance calculation device which calculates a white balance coefficient on 
the basis of the integration result by the white balance calculation, or that the image 
processing is performed in accordance with the white balance coefficient calculated by 
said white balance calculation device. 

Taniguchi et al. is similar to Nakamura et al. in that Taniguchi et al. teaches 
performing white balance (column 1, lines 9-15) on image data stored in a picture 
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memory (12, figure 1, column 9, lines 42-60). Taniguchi et al. also similarly teaches of a 
white balance integration device (15, 16, figure 1, column 9, line 63 through column 10, 
line 8). 

However, in addition to the teachings of Nakamura et al. and Anderson, 
Taniguchi et al. teaches of a white balance calculation device ("white balance coefficient 
calculating unit", 22, figure 1) which calculates a white balance coefficient on the basis 
of the integration result by the white balance calculation (See figure 1 , column 10, lines 
45-60. The white balance calculation device (22) calculates a white balance coefficient 
according to a plurality of factors determined in units 17-21 of figure 1, based upon the 
white balance integration of the white balance integration device (15,16). See also 
column 10, lines 6-44.), and that the image processing is performed in accordance with 
the white balance coefficient calculated by said white balance calculation device (See 
14, figure 1, column 9, lines 50-60, column 10, lines 56-60. Image processing is 
performed by the white balance adjusting unit (14) based upon the white balance 
coefficient calculated by the white balance coefficient calculating unit (22).). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the image processing device taught by the 
combination of Nakamura et al. and Anderson comprise a white balance calculation 
device and perform white balance processing based upon a calculated white balance 
coefficient as taught by Taniguchi et al. for the benefit of performing a sufficient degree 
of white balance adjustment appropriate to a colored picture without any erroneous 
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adjustment or adverse influence due to a high chromaticity region of the colored picture 
(Taniguchi at al., column 2, lines 40-50). 

The combination of Nakamura at a!., Anderson, and Taniguchi et al. teaches that 
said white balance calculation device (taught by Taniguchi et al.) calculates the white 
balance coefficient for the second RAW data (see above rationale) after said display 
device starts to display the live view image. Anderson teaches that parallel processing 
is performed by utilizing two buffers, column 6, lines 47-56. Anderson teaches that the 
image data from one buffer is displayed on the LCD (402) as the image data from the 
other buffer is processed, column 6, lines 47-56. Therefore, an object image is 
continuously displayed on the display, including any time when a white balance 
coefficient is being calculated. 

21 . Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,619,347) as applied to claim 16 above, and further in view of Sasaki (US 
6,961,085). 

Consider claim 17, and as applied to claim 16 above, the combination of 
Nakamura et al., Anderson, and Taniguchi et al. teaches that said display device 
displays the object image after said white balance calculation device calculates the 
white balance coefficient (See claim 16 rationale. Anderson teaches of a continuous 
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display.). However, the combination of Nakamura et al., Anderson, and Taniguciii et al. 
does not explicitly teach a defect correction device. 

Sasaki is similar to Nakamura et al. in that image data is collected from the 
image sensor (12, figure 1), preliminary processing is performed to yield first image data 
(see step 41 , figure 7) which is written into a buffer memory (26a, figure 7). Sasaki also 
similarly teaches that the first image data is read from the buffer memory (step 42, 
figure 7) for additional processing (see column 10, lines 1-57). 

However, in addition to the combined teachings of Nakamura et al., Anderson, 
and Taniguchi et al., Sasaki teaches that the apparatus further comprises a defect 
correction device which corrects a defective pixel portion of image data when the image 
sensing element has a defective pixel (See column 10, lines 21-57. Sasaki teaches that 
the locations of defective pixels are stored in memory, and when reading the data 
output from the buffer (26a), the defective pixels are corrected.), and that a control 
device controls said defect correction device in such a way that said defect correction 
device corrects a defective pixel portion of the image data (See column 10, lines 29-31, 
lines 37-41 .). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Sasaki correct defective pixels in the image processing device 
during the display of the object image as taught by the combination of Nakamura et al., 
Anderson and Taniguchi et al. for the benefit of keeping the influence of a defective 
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pixel to a minimum and preserving a liigh-definition image (Sasaki, column 3, lines 1-5, 
lines 12-16). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on (571) 272-7564. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

AC 

/Sinh Tran/ 

Supervisory Patent Examiner, Art Unit 2622 



